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A new paradigm in second language pedagogy has Computer Assisted Language Learning 
(CALL) playing a significant role. Much of the literature to-date claims that CALL can have 
a positive impact on students’ second language acquisition (SLA). Mixed method of research 
produces data to investigate if CALL positively affects student language proficiency, 
motivation and autonomy. Classroom observation of participants in their natural environment 
is a qualitative technique used but has situational variables that could skew results if not 
structured. A questionnaire is a quantitative tool that can offer insight regarding participants’ 
perception of perfonnance but can contradict what the researcher has observed. This paper 
will take an in-depth look at variables such as: instructor’s pedagogical application; blending 
CALL into the curriculum; types of CALL implemented; feedback received and their 
implications for design of the data collection tools. 


Introduction 

A new paradigm in teaching English to second language learners (L2) has evolved over the 
last 40 years which essentially revolves around the technological advancements within 
education itself. Development and application of Computer Assisted Language Learning 
(CALL) has grown from the behaviorist theory of learning which allowed for extensive drills, 
explicit grammar instruction and translations, to a process of discovery and expression that is 
an extension to communicative theory as it now allows L2 learners to converse in a more 
meaningful authentic context (Singhal, 1998; Lee, 2000, Warschauer, 2004). Nagata (1996) 
reviewed various comparative studies which were done in the 1980s and 90s on second 
language acquisition (SLA) and found that in most cases there were no significant differences 
in student learning. However, much of the literature to-date claims that using CALL will 
have a positive effect on students’ SLA. their motivation and autonomy. 
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Studying effects of CALL require using both qualitative and quantitative data collection tools 
such as observations of participants in their natural environment and questionnaires eliciting 
their opinions/reactions. Therefore, the researcher needs to be cognizant of the situational 
variables that occur in an uncontrolled observation. Petersen and Dutton (1975) first 
recognized that variables in research common to the classroom are easily neglected. 
Situational variables such as the instructors’ pedagogical knowledge and application; the 
application of CALL during instruction and the types used; the kind of feedback, both 
computer and instructor, the student receives while using CALL; and finally attitude and 
motivation, both student and instructor, towards CALL whether it is pre-conceived or 
developed in the classroom; all need to be acknowledged by the researcher in order to 
produce valid and reliable results. This paper will begin by discussing the aforementioned 
situational variables. It will then look at the implications of these variables when designing 
data collection tools. 

Literature review 

Pedagogical knowledge and application 

Many second language theorists believe that interaction is the most important way for 
learners to obtain infonnation needed for language learning (Chapelle, 1997; Podcameni & 
Salies, 2001; de la Fuente, 2003; Gonzalez-Lloret, 2003). Interactionist learning is a 
methodological approach to SLA that uses language as a tool to create and maintain social 
relations (Richards & Rodgers, 2001). L2 learners need to be active when engaging in 
communicative activities to understand what they can or cannot do. Students’ interaction 
among peers, with instructors or on the computer allows them to negotiate meaning (de la 
Fuente, 2002; Ariza & Hancock, 2003) that leads to comprehension of the meaning 
(Chapelle, 1997; Murphy, 2007). It is believed acquiring language is based upon receiving 
messages that can be made comprehensible through a variety of strategies, such as linguistic 
simplification, use of realia, visuals, pictures, graphic organizers and computer assisted 
language learning (Ariza & Hancock, 2003; Gonzalez-Lloret, 2003; Banados, 2006; 
AbuSeileek, 2007; Blake, 2007; Coyrell & Chlup, 2007). 

Embedded in the interactionist learning theory is collaboration which takes an important role. 
Language learners require comprehensible input in order to negotiate meaning and 
understanding in the target language (de la Fuente, 2003; Coryell & Chlup, 2007). 
Collaborative activities provide learners with the opportunity to share and construct 
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knowledge allowing them to realize the value in the activity while learning new skills. 
Working on skills cooperatively presents a more authentic interaction as learners exchange 
language input/output clarifying meaning and transferring knowledge. 

Interaction not only involves student to student/student to teacher exchanges but also between 
student and content. In a review done by Singhal (1998) on computer use in the classroom, 
she identified that researchers have created models to describe the interaction between the 
reader/text. She asserts that the most effective and successful use of technologies are those 
that generate students’ participation. Technology needs to allow for the flow of information 
so a student does not get trapped into a particular sequence but instead allows for flexibility; 
hence, the need for instructors to understand the most beneficial use of computers. 

Blending Computer Assisted Language Learning into the Curriculum 

Experienced educator, W. James Popham (2008) recognizes that “teachers themselves are a 
widely divergent variable” (p. 16); that they are particularistic due to their idiosyncratic 
experiences. That being the case, particular types of CALL will be applied in the classroom 
based on the instructors comfort and knowledge of language software. CALL is to be used 
as an aide to the presentation, reinforcement and assessment of material to be learned for the 
purpose of enhancing comprehension and attitude during SLA. Various applications include: 
on-line activities; commercial courseware; self-developed courseware; computer mediated 
communication (CMC) such as email, chat (written - MSN or verbal - Skype), video- 
conferencing, Blogs, videos and multimedia, word processors, electronic dictionaries. CALL 
will usually include a substantial interactive element to elicit student attention and motivation 
when learning a second language (Gulbahar & Madran, 2009; Stockwell, 2007) and in a 
classroom allows for experiential learning; permits interaction with authentic materials, can 
be individualized to suit the needs of the learner or group of learners; and motivates students 
and encourages independent learning (Lee, 2000). 

Blended learning is a pedagogical approach to SLA that combines face-to-face (F2F) 
instruction with the use of technology (Shanna & Barrett, 2007; Coryell & Chlup, 2007). Its 
flexibility when learning recognizes the benefits of training and assessment online while 
using other modes to elaborate on instruction (Banados, 2006) and feedback (Mandemach, 
2005; Murphy, 2007). Even with the long and evolved inclusion of CALL in L2 pedagogy, 
the concept of blended learning is a relatively new one. As such, there is an apparent gap 
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between practice and theory (Stracke, 2007). Ngu and Rethinasamy (2004) evaluated a 
CALL software that taught prepositions and found that when students were left on their own 
to tackle new commercial software to improve their language skills, they had to invest high 
mental effort to learn the CALL lesson; this and the fact that language is a social activity 
where the user needs to be able to respond to context and conditions are reasons that research 
suggests it is best done in a blended environment. 

Howard, Remenyi and Pap (2006), state that blended learning is concerned with effectively 
leveraging the strengths of differing kinds of learning activities to achieve the overarching 
learning objectives. Whether the instructor selects to use electronic dictionaries for 
vocabulary, computer mediated communication technologies for grammar or writing, or 
commercial software for speaking, it should not be considered as a worthwhile replacement 
for F2F instruction. Evidence has been provided in support of the assertion that when 
working on stand-alone applications learners can make poor decisions about directing their 
own learning or they become distracted and unable to complete exercises as they lack the 
feedback that forces them to think about their own errors (Fischer, 2007; Ngu & 
Rethinasamy, 2004; Murphy, 2007). Recent research challenges pedagogical assumptions 
made by CALL that reveals learners acknowledging that it is not a worthwhile replacement 
for classroom-based instruction but rather a balance between on-line access to knowledge and 
F2F human interaction (Gulbahar & Madran, 2009; Ayres, 2002) with the presence of their 
teachers to provide guidance and facilitation (Son, 2007). 

Given that an apparent gap can exist between theory and practice, implementation should not 
be done without ensuring that facilitation and communication remain. Kerres and de Witt 
(2003) offer a way of blending technology with traditional learning by giving attention to 
content of the materials; to communication between leamers/teacher, learners/peers; and to 
construction of knowledge. By keeping these three components in mind, the teacher can use 
CALL to facilitate learning in various ways. Although they suggest that in each component 
CALL be used, it should be blended through a mix and match approach with traditional 
learning. For instance, content of materials can be addressed through textbook related 
materials, and communication of materials can be achieved through the use of web-based 
activities. No matter the philosophy, one common denominator to language learning is that 
L2 learners need to interact to allow for the construction of new knowledge. 
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Feedback 

Research has identified various kinds of feedback both computer-mediated and human; 
corrective/verification or elaborative respectively. Feedback received from others helps an 
L2 learner to develop a better overview of how they are performing on a task and enables 
them to see what needs to be improved (Hwang & Arbaugh, 2009). Students can demonstrate 
significantly better performance in tenns of problem solving, conceptual understanding, 
transfer and retention of language when receiving feedback (Xu, Luetkehans, Hayall, & 
Smith, 2008). Podcameni and Salies (2001) view SLA as a process of feedback that involves 
the learner, interlocutor and external environment. They see the L2 learner as receiving input 
from the external environment and/or interlocutor; production of language is addressed in 
some way producing feedback and the learner assimilates the new information by 
reproducing the language. 

Elaborative feedback includes executing, showing, explaining and/or questioning that 
explains why the learners’ responses are correct or incorrect. Studies indicate it is a more 
effective fonn of feedback that influences student learning by developing a deeper conceptual 
understanding of relevant infonnation (Mandarnach, 2005; Murphy, 2007; Xu, Luetkehans, 
Hayall, & Smith, 2008; Wang & Wu, 2008) which is most effectively generated through 
human feedback. A F2F environment, offers L2 learners instant and elaborative feedback 
giving them the incentive they need to re-engage in their activity to enhance the opportunity 
to leam (Murphy, 2007). 

While most CALL does give feedback, as it is a design feature, it is generally verification 
feedback which does not necessarily give learners the metacognitive knowledge and skills 
that teachers and peers can provide on the spot (Mandernach, 2005). Much of the CALL 
feedback focuses more on grammatical or sentence structure and can be limited to 
spellcheckers, grammar checkers and discrete string or keyword matching (Blake, 2007). 
However, CALL is an effective way to make available a practice environment where students 
can think, reflect, and create language slowly (Pinkman, 2005) while receiving the support 
necessary to improve. 

Student attitude and motivation 

Computer technology in SLA can act as a source of stimuli by providing fun games and 
communicative activities reducing the learning stresses and anxieties (Lai & Kritsonis, 2006) 
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while allowing repeated attempts to succeed (Zapata & Sagarra, 2007). On the other hand, 
some L2 learners, particularly in lower levels lack the ability to direct their learning and can 
display chaotic behaviour towards the task indicating that beginning level students can 
become overwhelmed by the demands of the task (Fischer, 2007). These two situations can 
either have a positive or negative effect on both the student’s and instructor’s attitude toward 
using CALL to leam a second language. 

In a study done by Ayres (2002) concerning learner attitudes, he draws attention to the 
importance of making a connection between the CALL materials and the course taught to 
establish a positive student attitude. When developing pedagogy for an L2 classroom, 
research supports that CALL can provide an avenue for exposure to and interaction with 
authentic materials that enhance the learning experience (Lai & Kritsonis, 2006; Hsu, 2005; 
Pinkman, 2005). Nevertheless, the authenticity means selecting materials relevant to program 
requirements in order to keep students’ attitude enthusiastic and positive. 

During a study done by Son (2007) using a Web-based language learning activity, he noted 
that when his students were working, they displayed very few off-task actions, either on-line 
or off-line, pointing out that their level of interest kept them motivated to complete the tasks 
provided. Raby (2007) studied the motivation level of his students to see if the types of 
linguistic tools they used such as data banks, learning sites or CD-ROMS and cognitive tools 
such as online dictionaries, spelling correctors and storage applications increased student 
motivation. It was his conclusion that Infonnation and Communication Technology (ICT) 
did motivate his students in two ways: by opening up a new perspective for their language 
work and by increasing their autonomy because they were able to develop and control their 
work according to their own wishes. Warschauer (1996) studied L2 learners to ascertain their 
motivation when dealing with writing activities and surmised that all held positive attitudes 
toward using computers; they saw and felt the benefits of it thinking that it could help them 
learn better and more independently. 

Collection tools 

Data collection to detennine the effectiveness of CALL will take on a somewhat ethnographic 
approach to study. Ethnography entails getting involved in a social situation to find out how 
the participants view that world while the researcher describes how that culture operates 
(Goldbart & Hustler, 2005). Albeit the culture being studied in this case is that of the 
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classroom, each classroom will have a separate and individual learning community as the 
instructors will demonstrate particular idiosyncrasies and/or pedagogies. Observations are key 
to doing ethnographic research which means the researcher is present in a situation and makes 
a record of his/her impressions as the learning occurs. A significant intricacy in this situation 
is the complexity of human behaviour. Involved are the various interactions among 
participants, the varying pedagogies and the subjectivity of the researcher who is actively 
engaged in making sense of the behaviours and interpreting meaning of the observed events 
(LeCompte & Goetz, 1982; Jones & Somekh, 2005). Often what the participants’ 
construction of what is real and the interpretations of the observer are unlikely to match. 
Therefore, reliability in this situation becomes conditional on the design of the observation as 
it is concerned with replicability of scientific findings (LeCompte & Goetz, 1982) which 
lends itself to the validity of the research. While unstructured classroom observations can 
collect rich description of the student/teacher interaction in their natural environment, 
structured observations that include scheduling in advance, recognizing and categorizing 
situational variables that need to be noted in each classroom, would be more beneficial. As 
did both AbuSeileek (2007) & Son (2007), video-taping the class added to the thoroughness 
of data collected by producing authentic data. In any case, decisions need to be made in 
advance concerning the areas of focus to include students’ engagement with CALL, their 
initiative and attitude towards using CALL and the amount student/teacher interaction that 
occurs, as all are among behaviours to be noted. Since values not characteristics describe 
qualitative research a checklist for observation would ensure objectivity and validity. 

Attitude towards using CALL will be an important factor in detennining whether or not it has 
an effect on the learners’ proficiency, motivation or autonomy. In 1975, Petersen & Dutton 
first acknowledged the complexities involved with measuring participant attitudes. Chi- 
square test is one of the most commonly used techniques to explore relationships using 
nominal and/or ordinal data (Bames & Lewin, 2005). In his study concerning learner 
attitudes, Ayres (2002) drew attention to the importance of making a connection between the 
CALL materials and the course taught to establish a positive student attitude. He performed a 
chi-square test to determine if there were differences between the groups pertaining to 
motivation and perceptions of CALL. 

To elicit pertinent data for such analysis, a Likert style questionnaire can address self- 
evaluation through direct input from the participants; their feelings towards having used 
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CALL, and to render their attitudes toward a number of issues such as: the types of CALL 
used; their ease of using CALL; self-evaluation, etc. Many questionnaires have included 
open-ended questions which give the participants an opportunity to express more openly their 
attitude towards using CALL (Nagata, 1996; Pinkman, 2005; Hsu, 2005; Son, 2007). 

Implications for design 

Onwuegbuzie and Daniel (2003) recognized errors in both qualitative and quantitative 
research. They acknowledge that validity can be a concern as its definition in the educational 
community is ambiguous. Generally, validity is understood to be the “trustworthiness of 
inferences drawn from data” (Eisenhard & Howe, 1992, p. 644, as cited in Freeman, 
deMarrais, Preissle, Roulston, & St. Pierre, 2007). There is also claim that reliability of 
observation findings can be questionable and the infonnation gleaned from the observation 
must be trustworthy. Therefore, the educational community has adopted the use of 
triangulation to help reduce methodological errors in research (Onwuegbuzie & Daniel, 
2003). Commonly used in a mixed method of data collection is the combination of 
observations (qualitative) and questionnaires (quantitative) thus producing the triangulation 
necessary to produce trustworthy results. 

Given the situational variables to be encountered when studying the effects of CALL, the 
researcher is in a position of ensuring the data collection is reliable and valid. Ethnographic 
research is concerned with developing theoretical ideas rather than testing out existing 
hypothesis however it is unlikely the researcher will enter into the environment with a blank 
mind. Therefore, in designing the observations, initial behaviours should be noted as they 
produce specific questions that can validate the data collection by acknowledging the 
situational variables. The following questions can act as a foundation when developing 
structured classroom observations and be administered in each learning environment: 

• How is CALL being used in the classroom? How much instructional time is given to 
CALL? How is it blended into the curriculum? 

• Do the computer applications used in the classroom address all language skills or only 
specific skills? Are they web-based, commercial courseware, self-developed 
courseware, etc? 

• What type of feedback is used in the classroom: elaborative, corrective, directive, 
computer mediated feedback or a blend? 
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• How do students/instructors feel about using CALL? Does feedback influence student 
attitude/behaviour? 

Using these questions as a foundation, guidelines for gathering data can structure the 
observational instrument (Colvin, Flannery, Sugai, and Monegan, 2008). Colvin et al. (2008) 
identified three variables around which they built their observational tool to assist in ensuring 
the observations would be administered reliably and the resultant findings would be valid and 
useful to the participants. Nevertheless, the researcher must be open to problem 
reformulation as interaction in the classroom may alter at different times of the day and 
among particular teachers/students. Decision-making is constant and data collection and 
analysis are interrelated and on-going throughout most of the research process (Goldbart & 
Hustler, 2005). 

Questionnaires present their own set of issues and require structured design to include all 
aspects of participants’ involvement. It requires clear aims and objectives and be structured 
logically into section and subsections, if necessary, to filter questions so participants only 
respond to relevant questions. The questionnaire should include both closed and open-ended 
questions to ensure full disclosure of participants experience, attitude and level of learning. 
Quantitatively, data analysis can be attained through nominal data, ordinal data and interval 
data and are interpreted using descriptive statistics (Lewin, 2005). Qualitatively, descriptions 
of classroom behaviour can be attained through inductive reasoning, beginning with a specific 
set of observations and reaching conclusions based on the analysis of those observations. 
Values not characteristics describe qualitative research as reliable research seeks to ensure the 
interpretive, value-laden, contextual and contingent nature of social knowledge (Greene, 
Kreider & Mayer, 2005). 

As an aside to this discussion, it is imperative that informed consent be ascertained as 
observations involve invading other people’s space and constructing meanings from the 
experience of observing their interaction is done by the researcher, rather than the 
participants’ own accounts of their activities. Ethical issues can arise if the participants are 
not adequately infonned of the intentions of the research bringing into questions the validity 
of the results. 
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Conclusion 

This paper has discussed possible situational variables that should be acknowledged before 
entering into a classroom observation in order to maintain reliable and valid results. It 
suggests the need to structure the observations by using a checklist collection instrument; to 
use a closed and open-ended questionnaire to elicit student/instructor attitudes; and that using 
a mixed method of data collection such as this will triangulate the results creating trustworthy 
authentic data. Finally, in both instances of data collection which include participants, it is 
necessary to obtain their consent to include themselves willingly into the study. Taking all of 
these things into account will help to develop the necessary guidelines that will ensure valid 
and reliable results 
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